Hybrid Rounding Techniques for Knapsack 

Problems* 



Monaldo Mastrolilli and Marcus Hutter 
IDSIA, Galleria 2, 6928 Manno, Switzerland, {monaldo, marcus}@idsia.ch 

February 1, 2008 



Abstract 

We address the classical knapsack problem and a variant in which 
an upper bound is imposed on the number of items that can be selected. 
We show that appropriate combinations of rounding techniques yield 
novel and powerful ways of rounding. As an application of these tech- 
niques, we present a linear-storage Polynomial Time Approximation 
Scheme (PTAS) and a Fully Polynomial Time Approximation Scheme 
(FPTAS) that compute an approximate solution, of any fixed accu- 
racy, in linear time. This linear complexity bound gives a substantial 
improvement of the best previously known polynomial bounds . 



1 Introduction 

In the classical Knapsack Problem (KP) we have a set N := {l,...,n} 
of items and a knapsack of limited capacity. To each item we associate a 
positive profit pj and a positive weight Wj. The problem calls for selecting 
the set of items with maximum overall profit among those whose total weight 
does not exceed the knapsack capacity c > 0. KP has the following Integer 
Linear Programming (ILP) formulation: 

maximize Pj x j (1) 

subject to w j x j < c (2) 

^■€{0,1}, jeN, (3) 
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where each binary variable Xj, j G N, is equal to 1 if and only if item j is 
selected. In general, we cannot take all items because the total weight of the 
chosen items cannot exceed the knapsack capacity c. In the sequel, without 
loss of generality, we assume that YljeN w i > c anc ^ w i — c ^ or evei T J ' £ N. 

The It-item Knapsack Problem (kKP), is a KP in which an upper bound 
of k is imposed on the number of items that can be selected in a solution. 
The problem can be formulated as with the additional constraint 

(4) 

with 1 < k < n. 

KP has widely been studied in the literature, see the book of Martello 
and Toth jU] for a comprehensive illustration of the problem. kKP is the 
subproblem to be solved when instances of the Cutting Stock Problem with 
cardinality constraints are tackled by column generation techniques. kKP 
also appears in processor scheduling problems on computers with k pro- 
cessors and shared memory. Furthermore, kKP could replace KP in the 
separation of cover inequalities, as outlined in |2j. 

Throughout our paper let OPT denote the optimal solution value to the 
given instance and w(F) = J2jeF w j an< ^ v{F) = SjgF^i' wnere F C. N. 
An algorithm A with solution value z A is called a (1 — e)- approximation 
algorithm, e £ (0,1), if z A > (1 — e)OPT holds for all problem instances. 
We will also call e the performance ratio of A. 

Known Results It is well known that KP is NP-hard but pseudopoly- 
nomially solvable through dynamic programming, and the same properties 
hold for kKP 2 . Basically, the developed approximation approaches for KP 
and kKP can be divided into three groups: 

1. Approximation algorithms. For KP the classical ^-approximation al- 
gorithm (see e.g. jS]) needs only 0(n) running time. A performance 
ratio of ^ can be obtained also for kKP by rounding the solution of the 
linear programming relaxation of the problem (see [2]); this algorithm 
can be implemented to run in linear time when the LP relaxation of 
kKP is solved by using the method by Megiddo and Tamir 

2. Polynomial time approximation schemes (PTAS) reach any given per- 
formance ratio and have a running time polynomial in the length of 
the encoded input. The best schemes currently known requiring linear 
space are given in Caprara et al. [2j: they yield a performance ratio of 
e within 0(n^/ e ^ 2 + nlogn) and 0{n} l l £l{ ~ 1 ) running time, for KP 
and kKP respectively. 

3. Fully polynomial time approximation schemes (FPTAS) also reach any 
given performance ratio and have a running time polynomial in the 
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length of the encoded input and in the reciprocal of the performance 
ratio. This improvement compared to 1. and 2. is usually paid off by 
larger space requirements, which increases rapidly with the accuracy 
e. The first FPTAS for KP was proposed by Ibarra and Kim later 
on improved by Lawler (Hj and Kellerer and Pferschy [Jj . In Caprara et 
al. [2] it is shown that kKP admits an FPTAS that runs in 0(nk 2 /e) 
time. 

New Results Rounding the input is a widely used technique to obtain 
polynomial time approximation schemes [I]. Among the developed rounding 
techniques, arithmetic or geometric rounding are the most successfully and 
broadly used ways of rounding to obtain a simpler instance that may be 
solved in polynomial time (see Sections l2.1l and l2.2l for an application of these 
techniques to kKP). We contribute by presenting a new technical idea. We 
show that appropriate combinations of arithmetic and geometric rounding 
techniques yield novel and powerful rounding methods. To the best of our 
knowledge, these techniques have never been combined together. By using 
the described rounding techniques, we present a PTAS for kKP requiring 
linear space and running time 0(n + k- (l/e) ^ 1 /^). Our algorithm is clearly 
superior to the one in [2], and it is worth noting that the running time 
contains no exponent on n dependent on e. Since KP is a special case 
of kKP, we also speed up the previous result for KP to 0(n ■ (l/e)°^ 1//e -'). 
Finally we present a faster FPTAS for kKP that runs in 0(n + k/e 4 + 1/e 5 ) 
time and has a bound of 0(n + 1/e 4 ) on space requirements. 

2 Rounding techniques for kKP 

The aim of this section is to transform any input into one with a smaller size 
and a simpler structure without dramatically decreasing the objective value. 
We discuss several transformations of the input problem. Some transforma- 
tions may potentially decrease the objective function value by a factor of 
1 — 0(e), so we can perform a constant number of them while still staying 
within 1 — 0(e) of the original optimum. Others are transformations which 
do not decrease the objective function value. When we describe the first 
type of transformation, we shall say it produces 1 — 0(e) loss, while the 
second produces no loss. 

Let P H denote the solution value obtained in 0(n) time by employing 
the 1/2-approximation algorithm for kKP described in [2]. In [2], it is 
shown that 

2P H >P H + Pmax > OPT > P H , (5) 

where p max = m&xjpj. 

Throughout this section we restrict our attention to feasible solutions 
with at most 7 items, where 7 is a positive integer not greater than k. The 
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first observation is that at most an e-fraction of the optimal profit OPT 
is lost by discarding all items j where pj < eP H /j, since at most 7 items 
can be selected and P H < OPT. From now on, consider the reduced set of 
items with profit values greater than eP H /*y, with 1 — e loss. 

In order to reduce further the set of items, a useful insight is that when 
profits are identical we pick items in non-decreasing order of weight. Since 
the optimal profit is at most 2P H , for each fixed p € {pi, ...,p n }, we can 

keep the first n = min|7, | items with the smallest weights, and 

discard the others with no loss. Of course, we cannot hope to obtain a 
smaller instance if all profits are different. In the following, we show how 
the number of different profits can be reduced by rounding the original 
profits. We revise two rounding techniques and show that an appropriate 
combination of both yields to a better result. We call a profit value p large 



if p > 



2P t 



and small otherwise. 



2.1 Arithmetic rounding 

A sequence ai, 02, ... is called an arithmetic sequence if, and only if, there is a 
constant d such that en = a\ +d- (i— 1), for all integers i > 1. Let us consider 
the arithmetic sequence S a ("/) obtained by setting a\ = d = eP H /j. We 
transform the given instance into a more structured one by rounding each 
profit down to the nearest value in Sa('y). Since in the rounded instance 
the profit of each item is decreased by at most eP H /j, and at most 7 items 
can be selected, the solution value of the transformed instance potentially 
decreases by eP H . Of course, by restoring the original profits we cannot 
decrease the objective function value, and therefore, with 1 — e loss, we can 
assume that every possible profit is equal to dj = • i for some i > 1. 
Furthermore, since p mSLX = maxjgjvP? < P H > the number of different profits 
is now bounded by [ 7 £ P pfT j < |_e J ■ 

The largest number rii of items with profit dj, for i = 1, |_^J, that can 
be involved in any feasible solution is bounded by 



< min{7, 



OPT 

eP H - 



} < min{7, 



2l 



si 



}• 



and we can keep the first n, items with the smallest weights, and dis- 
card the others with no loss. Now, the number of items with profit dj 
is at most 7, if a% is a small profit (i.e. when i = 1,..., If J), an< ^ a ^ 



most 



otherwise 



= LIJ + LiJ)' Thus ' b y aPP^ing the de- 
scribed arithmetic rounding we have at most [2/e\ 7 items with small prof- 
I 2 I 

its and ^ — [ 2 J+1 ^ with large profits. Recall that when a summation 

can be expressed as Sfc=x/(^)j wnere fity 1S a monotonically decreas- 
ing function, we can approximate it by an integral (see, e.g. [S] p. 50): 
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f» +1 f(k)dk < YX=J(k) < ^_ l f{k)dk. Furthermore, we are assuming 
that < e < 1, and recall that ln(l + x) > x/(l + x), for x > — 1. Therefore, 
the total number of items in the transformed instance is bounded by 



7 + 



LiJ 

E 

=L!J+i 



27 



£?, 



< 



< 



2 2 
_ 7 + _ 7 



27 



(1 



m 
E 

=LSJ+i 



27 



(l + ln 7 -ln(2-e)) 



^-l 



< ^(1 +ln 7 ) = 0(Jln 7 ). 

We see that by applying the described arithmetic rounding we have at most 
2 7 /e items with small profits and ^ ln 7 with large profits. 

A natural question is to see if the provided bound is tight. Consider an 
instance having 7 items for each distinct profit • i, where i = 1, |_^J . 
Observe that by applying the described arithmetic rounding, we have exactly 



[2/eJ 7 items with small profits and ^ 



Li 



=L2J+i 



with large profits. What 



remains to be shown is to bound the number of items to be ln 7 ): 



2 

£ 


L e J 

7+ E 


2 7 

£1 









>- fr- + E(5-» 



21-1 

2 ^ 1 ,7 



> -(1 + 2 / £ ^) = 2(l +2 ln 7 -21n(2+£)) 
= 0(-ln 7 ). 



2.2 Geometric rounding 

A sequence a±,ci2, ■■■ is called a geometric sequence if, and only if, there is 
a constant r such that a, = aq • r i_1 , for all integers i > 1. Let us consider 
the geometric sequence 5 S ( 7 ) obtained by setting a\ = £P H and r = 
We round each profit down to the nearest value among those of S g ( 7 ) . Since 
cii = (1 — e)aj + i, for i > 1, each item profit is at most decreased by a factor 1 
of 1 — £, and consequently, the solution value of the transformed instance 
potentially decreases by the same factor of 1 — e. Therefore, with 1 — 2£ loss, 

lr This is true only for profits larger than oi; recall that we are assuming, with 1 — e 
loss, that all profits are larger than oi. 
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we can assume that every possible profit is equal to a« = ■ (-pM* 1 for 
some i > 1. Furthermore, since p max < P H , the number of different profits 
is bounded by the biggest integer /3 such that 
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Since ln(^-) > e, we have (3-l< ^Vfi < 
the largest number rij of items with profit a, , for z 



Mn^. 

e e 



In any feasible solution, 
= 1, ...,/?, is bounded by 



rij < min{7, 



OPT 



eP H . ( i y;-i 

7 U-e/ 1 



} < min{7, 



}■ 



and we can keep the first rij items with the smallest weights, and discard 



the others with no loss. Let a = 



lELg/gJ 



In- 



+ 1. Again, the number of items 



with profit a>i is at most 7, if ai is a small profit (i.e. when 1 < i < a), 
and at most 



-^r(l — e) 1 otherwise (i = a + 1, ■■■,(3)- Therefore, the total 
number of items in the transformed instance is bounded by 



«7 



+ £ 



i=a+l 



2 7 



(1-E) 



<(-ln(2/e) + l) 7 +^ 

£ £ 



0(1 1„ i; 



Moreover, we can easily show that this bound is tight. Consider an instance 



having 7 items for each distinct profit ■ (jz^Y 1 , where i = 1, ...,/?. 
By applying the described geometric rounding technique, we have exactly 
07 items with small profits and Yli= a +i T"(l ~~ with large profits. 

What remains to be shown is to bound the number of items to be flQ In i): 



«7+ 



2 7 



(i- £ y 



> 



> 



In 12/eJ 
«7 > — M— L 7 



7" 



O(^ln-). 

e £ 



We see that by applying the geometric rounding we have at most 7/e 
items with large profit, while O(jln^) items with small profits. Contrary 
to arithmetic rounding, the set of items that has been reduced most is the 
set with large profits. This suggests us to combine the described rounding 
techniques as described in the following subsection. 



2.3 Parallel Arithmetic & Geometric rounding 

We use arithmetic rounding for the set of items with small profits and ge- 
ometric rounding for large items. Let us say that these two techniques are 
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applied in "parallel". More formally, let us consider the hybrid sequence 
Sagi^) = (iijQ2) •••) defined by setting 



eP H 



{ 



i for i = 1,..., [2/eJ 

(-±-)a+i-l2/e}-i otherwise. 
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We round each profit down to the nearest value among those of 5^(7). 
Now, consider each set Zi of items with the same rounded profit value a^, 
and take the first 



items with the smallest weights we get in Zi. Selecting the first rii items 
with the smallest weights can be done in 0(|Zj|) time. That is, 0(|Zj|) time 
is sufficient to select the rij-th item with the smallest weight (see ^) and 
only 0(|Zj|) comparisons are needed to extract the nj — 1 items with smaller 
weights. Therefore the amortized time is linear. 

By using the arithmetic rounding technique for small items, we have at 
most 27/e small items with 1 — e loss (see Section l2~Tj) . While, by using 
the geometric rounding technique described in Section 12.21 for large items, 
we have at most 7/e large items with 1 — e loss. The resulting transformed 
instance has at most 37/e items with 1 — 2e loss. Furthermore, let ip = 
(3 — a + \2/e\ + 1. Observe that the tp-th element of S ag {^) is larger than 
P H , i.e. &0 > P H . Consider any subset S C N of items with at most 7 
items, and let X{ denote the total number of items from S with profit in 
interval [aj,Oj-j-i), i = 1,2, ...ip — 1. Let us call vector (xi,X2, an 
S -configuration. It is easy to see that by using the reduced set of items it is 
always possible to compute a solution having the same (^-configuration, as 
any set SCAf with 7 items. Summarizing: 

Lemma 1 For any positive integer 7 < k, it is possible to compute in linear 
time a reduced set iV 7 C N of items with at most 37/e items, such that, for 
any subset S C N with at most 7 items, there exists a subset S 7 C such 
that S-y is the subset of N having the same configuration as S and with the 
least weights. 



Corollary 2 For any subset S C N with at most 7 items, there exists a 
subset 5 7 C Ny withw(S y ) < w(S), |5 7 | = \S\ andp{S 1 ) > p(S) — 2e-OPT. 

3 An improved PTAS for kKP 



Our PTAS for kKP improves the scheme of Caprara et al. 2 , and in fact it 
strongly builds on their ideas. However, there are several differences where 
a major one is the use of two reduced sets of items instead of the entire 





otherwise 



7 



set N: let £ := min{[l/e] — 2,/c}, where e < 1/2 is an arbitrary small 
rational number; our algorithm uses sets N k and Ng computed by using the 
Arithmetic & Geometric rounding technique (see LemmaEJ when 7 := k 
and 7 := £, respectively. 

For any given instance of kKP, the approximation scheme performs the 
following five steps (S-l)-(S-5). 

(S-l) Initialize the solution A to be the empty set and set the corresponding 
value P A to 0. 

(S-2) Compute the reduced sets N k and Ng. 

(S-3) Compute P H , i.e. the solution value returned by when applied to 
the whole set of instances. 

(S-4) Consider each L C Nt such that \L\ < I. If w(L) < c, consider 
sequence S ag {£) = (01,02,...) and let h be the smallest integer such 
that mhxj^LPj < o-h+i (assume mhij^LPj = if L = 0). Apply 
algorithm iJ5 to the subinstance S defined by item set {i £ Nk\L : 
Pi < a/1+1}, by capacity c — w{L) and by cardinality upper bound k — t. 
Let T and P H (S) denote the solution and the solution value returned 
by H2 when applied to S, respectively. If p(L) + P H (S) > P A let 
A := L U T and P A := p{L) + P H (S). 

(S-5) Return solution A of value P A . 

Observe that in step (S-4), subsets L are computed by considering just 
the items from N#. On the other hand, in step (S-4), we remark that the 
subinstances S are defined by using items from N^. 

3.1 Analysis of the Algorithm 

Step (S-2) can be performed in 0(n) time by Lemma ^ Step (S-3) runs 
in 0(n) time 0. In step (S-4) the algorithm considers 0{\N^\ + \Ni\ 2 + 
... + \Nf\ ) = 0(\Nt\ ) subsets. For each L the definition of subinstance S 
requires 0(|A^| • £) time. Algorithm applied to subinstance S runs in 
0(\S\) = 0(\N k \) time 0. By Lemmad \N k \ = 0{k/e) and \N e \ = 0(£/e). 
Therefore, step (S-4) is performed in 0(\N e \ £ ■ \N k \ ■ £) = 0(k ■ (f) m ) = 
k ■ (l/e)° {l / £ \ It follows that the overall running time of the algorithm is 
0(n + k ■ (l/e) 0(1/e) ), and it is not difficult to check that steps (S-l)-(S-5) 
require linear space. What remains to be shown is that steps (S-l)-(S-5) 
return a (1 — 0(e))-approximate solution. 

Consider sequence S ag {£) = (01,02,...). Let {ji, j£, ...} be the set of 
items in an optimal solution ordered so that pj 1 > ... > Pj e > and let 
A < £ be the largest integer such that pj x > a\ (if there is no item in the 
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optimal solution with profit > a\ then set A = 0). Let L* = {ji, j\} be 
the subset (possibly empty) obtained by picking the first A items with the 
largest profits. Consider subinstance S* defined by item set 



j 5 . ={ie N\L* 



Pi < mmj eL * pj if L* ^ 
Pi < ai, otherwise. 



by capacity c — w(L*) and by cardinality upper bound k — A. Clearly, 

p(L*) + OPT s * =OPT, (6) 

where OPTs* denotes the optimal value of instance S*. Now, consider the 
reduced set Nk and subinstance SI defined by item set 



is. 



I. 



{i G N k \L* 



Pi < minjei* pj if L* ^ 
Pi < ai, otherwise. 



(7) 



by capacity c — w(L*) and by cardinality upper bound k — A. By Corollary 
we have 

OPT s * > OPT s * - 2eOPT, (8) 

where OPTs* denotes the optimal value of instance S%,. 

Let us use L to denote the set of items having the same configuration as 
L* and the least weights. By LemmaQ m one of the iterations of step (S-4), 
set L is considered. In the remainder, let us focus on this set L, and consider 
the corresponding subinstance S defined in step (S-4). By Corollary |2J we 
have 

p(L)>p{L*)-2eOPT. (9) 

We need to show that the optimal solution value OPTs of instance S cannot 
be smaller than OPTs* 



k 



Lemma 3 OPT s > OPT 



k 



Proof. Assume L having the same configuration as L*. Recall that the 
subinstance S is defined by item set Is = {i € N^L : pi < a^+i}, where 
ah + i is the term of sequence S ag {l) = (01,02, •••) such that h is the smallest 
integer with mmj^L pj < a^+i (see step (S-4)). On the other hand, the 
subinstance S%. is defined by item set Ig* (see ®). If L* = then S = S%. 
and the claim follows. 

Otherwise (L* ^ 0), since we are assuming that L has the same config- 
uration as L*, there are no items from L* with profit in intervals [ai,ai + i), 
for i < h. Therefore, we have mmj^L* pj > ah and {i € N^\L : pi < 
a h} = {i € Nk\L* : pi < a^}. Furthermore, since there is at least one item 
from L with profit in interval [a/i> a /i+i) (recall we are assuming L* ^ 0), 
and since L* has the same configuration as L, there exists an item from L* 
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with profit pj < at+i and, therefore, naiXj^L* Pj < a-h+l- It follows that 
I s * C{iG N k \L* : Pi < a h+1 }. 

By the previous arguments, the items of S%, except those belonging to 
Ah = {i £ Nk Pi L : ah < Pi < a/1+1}, are also items of S, i.e., 

Is* CI s uA h . 

If there exists an optimal solution for St such that no one of the items 
from Ah is selected, then OPTs > OPTs* , since the knapsack capacity of 
S£ is not greater than the one of S, i.e. c — w(L) > c — w{L*) (recall that L 
is the subset having the same configuration as L* with the smallest weights) . 

Otherwise, let G± be the subset of items from Ah in an optimal solution 
for S%, and let g := |£?i|. Let G2 be any subset of {i € L*\L : ah < p% < 
a-h+i} containing exactly g items. It is easy to see that G2 exists (recall that 
L and L* have the same configurations and Ah C L). Furthermore, since 
G2 C L* and G\ C J5*, we have 

minp,- > rnaxp,-. (10) 

jeG 2 j'eGi 

Observe that w{L*) — w(L) > w(G2) — w{G\). Therefore, the knapsack 
capacity c — iu(L) of 5 cannot be smaller than c — w(L*) + w(G2) — w(G±). 
The solution G12 obtained from the optimal solution for St by replacing the 
items from G\ with those from G2 , requires a knapsack of capacity bounded 
by c — w(L*) + w(G2) — w{G\). Therefore, G12 is a feasible solution for 
S since the capacity of S is greater than the capacity of by at least 
— w{G\). Finally, from inequality (|Tn|) . the solution value of G12 is 
not smaller than OPTc* and the claim follows. ■ 

Let P H (S) denote the solution value returned by Hz when applied to 
S. Then we have the following 

Lemma 4 p(L) + P H (S) > (1 - 4e)OPT. 

Proof. Observe that by Lemma |31 and inequality we have 

OPT 5 > OPT 5 . - 2eOPT. (11) 

We distinguish between two cases. 

1. If p(L*) > (1 - e)OPT then by inequalities ©, ®, © and dJ, we 

have 

p(L) + P H (S) > P (L*)-2eOPT + ^OPT s 

> (1 - e)OPT - 2eOPT = (1 - 3e)OPT. 
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2. If p(L*) < (1 — e)OPT then each item profit in S* is smaller than 
— ^ OPT . Indeed, if A = I then the smallest item profit in L* , and 
hence each item profit in S*, must be smaller than — ^ OPT (other- 
wise p{L*) > (1 — e)OPT); else (A < £) by definition of A, there are at 
most A items with profits not smaller than a\ and therefore, each item 
profit in S*, must be smaller than a x = \P H < jOPT < ^-j^-OPT 
(since e < 1/2). Now, we claim that the largest profit in S is at 
most OPT + ~~f~' Indeed, since by definition of h we have 
a h < ^OPT < (1-e)^- < (LfJ - 1)^, it turns out that 
h < [~J — 1, and by definition of S ag (£), we have that ah+i = ah + ^j-- 
Therefore, for each item j belonging to S, profit pj is bounded by 

Since OPTs — P H (S) < maxj^sPj (see inequality @), we have 
OPT 

p(L)+P H (S) + — r > p(L) + OPT s 

> p(L*) + OPT s * - 4e • OPT = (1 - Ae)OPT. 



By the previous lemma, steps (S-l)-(S-5) return a solution that cannot 
be worse than (1 — 4e)OPT. Thus, we have proved the following 

Theorem 5 There is an PTAS for the k-item knapsack problem requiring 
linear space and 0(n + k ■ (l/e)°( 1//£ )) time. 

To compare our algorithm with the one provided in [2] notice that the 
running time complexity of the latter is 0(n^ 1 / £ l _1 ), whereas our scheme 
is linear. As in [2], our algorithm can be easily modified to deal with the 
Exact k-item Knapsack Problem, that is a kKP in which the number of 
items in a feasible solution must be exactly equal to k. The time and space 
complexities, and the analysis of the resulting algorithm are essentially the 
same as the one described above. Compare also with the general problem 
solver developed in p], where the elimination of a multiplicative constant 
(here 1/e) led to a larger additive constant (here (l/e)°( 1//e )). 



4 An improved FPTAS for kKP 

The main goal of this section is to present a different combination of arith- 
metic and geometric rounding techniques. Moreover we propose an improved 
fully polynomial time approximation scheme that runs in 0(n + /c/e 4 + l/e 5 ) 
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time. First we discuss separately the different steps in details, then we state 
the main algorithm and summarize the results in Section [4.21 

We start partitioning the set of items in two subsets £ = {j : pj > eP H } 
and S = {j :pj < eP H }. Let us say that C is the set of large items, while 
S the set of small items. Observe that the number of large items in any 
feasible solutions is not greater than A = min {k, [2/eJ }, since OPT < 2P H . 

4.1 Dynamic programming for large items 

In principle, an optimal solution could be obtained in the following way. 
Enumerate all different solutions for items in £,, i.e., consider all different 
sets U C £ such that w(U) < c and |J7| < k. For each of these U, compute a 
set T C S such that w(T) + w(U) < c, \U\ + \T\ <k and p(T) is maximized. 
Select from these solutions one with the largest overall profit. One of the 
problems with this approach is that constructing all possible solutions for 
items in C would require considering re ' 1 / 6 ) cases. To avoid the exponential 
dependence on 1/e (our aim is to obtain a fully polynomial approximation 
scheme), we will not treat separately all of these solutions. We begin with 
the description of a basic procedure that generates a list of all "interesting" 
feasible combinations of profit and number of selected large items. Each such 
combination is represented by a pair (a, I), for which there is a subset of items 
U C C with p{U) = a, \U\ = I and w(U) < c. Moreover w(U) is the smallest 
attainable weight for a subset of large items with profit at least equal to a 
and cardinality at most I. This list of all "interesting" feasible combinations 
is computed by using a pseudopolynomial dynamic programming scheme. 
Clearly, an optimal solution can be computed by using only the subsets U 
of large jobs associated to each pair (a, I). The time complexity will be then 
reduced, with l — 0(e) loss, by applying arithmetic and geometric rounding 
techniques, as described in Section 14.1.31 

Let a be the number of large items, and let (3 denote the number of all 
distinct feasible solution values obtained by considering only large items, i.e. 
(5 is the size of set 

V = {p(U)\U C C and w(U) < c and \U\ < k} . 

A straightforward dynamic programming recursion which has time complex- 
ity 0(a(3\) and space complexity 0(X 2 (3) (see 2 ), can be stated as follows. 
Let us renumber the set of items such that the first 1, \L\ items are large. 
Denote by function giia, I) for i = 1, \L\, a & V, I = 1, A, the optimal 
solution of the following problem: 

i 

gi(a, I) = min WjXj : 



Ylj=iPj x j 

= 1 X j ~ 



a: 



X 



€{0,l},i = l, 



12 



One initially sets go(a, I) = +00 for all I = 0, A, a € V, and then go(0, 0) = 
0. Then, for i = 1, |L| the entries for gi can be computed from those of 
by using the formula 

1 n ■ / 9i-i{a,l), 
gAa, I) = mm < ) , -r , 1 

[ 5i_i(a — pj, / — lj + itfj 11 / > (J ana a > pi 

Since /3 = 0(P H ) the described dynamic programming algorithm is only 
pseudopolynomial. In order to reduce the time complexity, we first prepro- 
cess large items by using a combination of arithmetic and geometric round- 
ing techniques, then we apply the above dynamic programming scheme. We 
start analyzing the two rounding techniques separately, then we show how 
to combine them. 



4.1.1 Geometric rounding 

The time complexity of the described dynamic programming can be reduced 
by decreasing the number a of large items and the number j3 of distinct 
solution values. 

We observed in Section [2] that if we want to reduce as much as possible 
the number of large items it is convenient to use geometric rounding. Con- 
sider the geometric sequence ^(7) described in Section 12^21 By applying 
the geometric rounding technique with 7 = A, the number a of large items 
can be reduced from 0(n) to 0(1/ s 2 ) with 1 — e loss. 

The next step is to compute the number of possible solution values after 
geometric rounding, i.e. the cardinality (3 of set V after that all profit 
values of large items have been geometrically rounded. The main result of 
this section is stated as follows. 

Theorem 6 The number of solution values after geometric rounding can be 
exponential in - . 

By the above theorem it follows that the running time of dynamic pro- 
gramming after geometric rounding is a constant that may depend exponen- 
tially on ~. Therefore, to avoid this exponential dependence on i, we will 
look at other rounding techniques. 



Proof of Theorem |HJ In the remaining part of this subsection we prove 
Theorem El The goal is to derive a lower bound on the number of possible 
solution values after geometric rounding, i.e. a lower bound on |V|. Recall 
that we defined the geometric sequence a, = ^j-(j^T 1 i n Section POI 
and here we assume that 7 = A. We focus on worst-case analysis. With this 
aim let us consider an instance I that after geometric rounding has at least 
\P H /aA items for each distinct profit value aj. Moreover, we assume that 
Wj = pj for every j € £, c = P H and k > 1/e. By definition of instance / 
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we see that every subset U with p(U) < P H is a feasible solution. Indeed, 
we have w(U) < P H = c and \U\ < p(U) /(min j^cPj) < V s — ^- ^ * ne 
previous arguments we see that |V| is bounded by below by the number of 
solution values y = J2ilo c i a i+i < -f with c = (cq,c\,...) G -2Vg°, where 
IV,^ is the set of sequences with non-negative integer components. Inserting 
a-i and e = j^p- this is equivalent to 

oo 

J2cie'(l+e'Y < X(l+e'). (12) 

i=0 

Clearly 

oo 

^Q £ '(l + £')'<!• (13) 

i=0 

implies (|T2*|). since A(l + e') > 1. 

For simplicity of notation we replace e' with e, and we focus on the 
cardinality of sets of the form 

r d-l 
R d e := \ V < 1 : V = £>e(l +e)*, c G W Q 

I i=0 

where Mq is the set of d-dimensional vectors c = (cq, c\, Cd-i) with non- 
negative integer components. It is more easy to find lower bounds on the 
set of vectors 



d-l } 
S'! : |CG^:^^(1 + £ )'<1 ill) 

i=0 J 



itself. To consider \S^\ instead of \R^\ is justified by the following Lemma. 



Lemma 7 For rational and for transcendental e > 0, the sets and S° 

have the same cardinality, i.e. the mapping f : S^: — > with /(c) 
vi-1 



Yli=o c ? e (l + e T i- s one-to-one and onto. 



Proof for transcendental e. 

d-i 

y = y 4$ °i x% = with hi := Ci — G Z and x := 1 + e 

i=0 

A real number is said to be transcendental if it is not the root of a polynomial 
with integer coefficients. For transcendental e also x is transcendental, which 
implies 6j = 0. Hence, y = y' implies c = c'. Obviously c = c' implies y = y'. 
This proves that S£ has the same cardinality as R^ for transcendental e. 

Proof for rational e. Assume by contradiction that there are two 
different vectors c/c'€ Mq with same solution value y = y' . Let n be the 
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largest index i such that a 7^ c-. Furthermore, let e = | — 1 > be rational 
with p > q £ IN having no common factors. With b := c — c' we have 

d-1 

= [y-y] = E 6 i £ ( 1 + £ ) < 

i=0 

integer 



n-1 



eTbA = ± 

i=0 * * i=0 



Since the last term q ^[•••] is a multiple of q, y — y' can only be zero if also 
b n p n is a multiple of q. With p also p n has no common factor with q, hence 
b n must itself be a multiple of q. The sum in (|14j) can only be less than 1 if 
each term is less than 1, i.e. Cje(l + e) J < 1. This implies 

Ci < e _1 (l + < for all i. (15) 

Together we get 

1 q 

< Ch) < - = < <? => |6 n | = |c n - < q => b n = => c n = c' n 

e p — q 

which contradicts our assumption c n 7^ c' n . Hence, y = y' implies c = c'. 
Again, that c = c' implies y = y' is obvious. This shows that S e has the 
same cardinality as R £ for rational s. ■ 

We don't know whether Lemma[3also holds for algebraic e. The follow- 
ing Lemma lower bounds S£°. 

Lemma 8 \S™\ > Ce B / £ with 5 = 0.3172... and C = 0.3200.. . 

Proof. From (|15|) we see that all Cj are zero for too large i (q = 
OVi > d max := rS^D- This shows that |S*| < |,S E 2 | < ... < \S d — \ = 
\S dmax+l \ = ... = \S%°\. The main idea in the following is to relate S d to 
the volume of a <i-dimensional simplex with volume larger than Ce B l £ for 
suitable d. 

We define a d-dimensional subset U d C M d , which is the disjoint union 
of unit cubes [co,co + 1) x ... x [cd-i,Cd-i + 1) for every c G S d . This set 
can be represented in the following form 

U d := jr€[0,cx)) d :^Ml + £y <lj 

The volume Vol(U d ) coincides with the cardinality of set S d since each point 
in S d corresponds to exactly one unit cube in U d , each having volume 1. 
Furthermore, let us define the ci-dimensional (irregular) tetrahedron 

T d e := jr€[0,oo) d : J> e (l+ £ r<lj 
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Obviously T £ d C U*, since [r»J < n . So we have \Sf > \ > \S%\ = Vol(U*) > 
Vol(T^). The tetrahedron is orthogonal at the vertex r = 0. The edges 
r = (0, 0, rj, 0, 0) have lengths [e(l + e)*] -1 , i = 0...d — 1. Hence, the 
volume of the tetrahedron is 

d-1 

Vol(T E d ) = i- JJ^l + e)*]- 1 = [^e d (l + £) d(d_1)/2 ] -1 
' i=0 

> e [-((fc)ln(de)-|((fe) a ]/ e = e f(de)/e 

with f(x) := —x lnx — \x 1 . In the inequality we replaced d — 1 by d and 
used d! < d d and 1 + e < e e . The best bound is found by maximizing 
e f{zd)/E sff.tX. d, or equivalently by maximizing f(x) w.r.t. x. We have 
-f'(A) = \xxA + 1 + A = for A = 0.2784.... Hence, d should be chosen as 
but since d is integer we have to round somehow, for instance d = |_— J. 
Note that \S^ \ increases with d, but our approximation becomes crude for d 
near d max . This is the reason why the maximizing d is less than d max . For 

small ewe have /(eLjJ) ~ /O7) = /(^) =-4 ln.4-^ 2 =A(1 + £A) =: 
B = 0.3172... with corrections of order O(e). This establishes an asymptotic 
bound ~ e B l s . More exactly, one can show that /(e|_g J) > ~~ 3) f° r 

all sr. This yields the bound 

|5~| > maxVol(T e d ) > e /(j4)(1- * )/e = Ce B/e with C = e~ B/A = 0.3200... 

■ 

The coefficient B can be improved to 0.7279... for sufficiently small e by 
using the more accurate Stirling approximation for d\. 

Using Lemma and |B1 it is now easy to lower bound the number of 
possible solution values for geometric profit distribution. ^From Lemma |S] 
we know that (|13|) has at least Ce B / e solution vectors c and from Lemma 
that (fT2*)) has at least Ce B l £ solution values y for rational e, and the proof 
of Theorem El follows. 

4.1.2 Arithmetic rounding 

Alternatively, we may think to apply arithmetic rounding to the set of large 
items. Let us consider the arithmetic sequence £0(7) described in Section 
12.11 By applying the arithmetic rounding technique with 7 = A, we observe 
the number of large items can be reduced to be bounded by 0(-p In |) with 

1— e loss. Moreover each element of set V is equal to i for some i = A, A+ 
1, 2 [X/e\ . It follows that the size of set V is bounded by 0(l/e 2 ), and the 
overall time of the dynamic programming algorithm is now 0{\ In We 
see that in comparison to the geometric rounding and although the number 
of large items is larger, the arithmetic rounding technique is able to reduce 
much more the size of set V . However and again, we can take advantage 
from both techniques by combining them as described in the following. 
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4.1.3 Serial Geometric & Arithmetic rounding 

We first apply geometric rounding with 1 — e loss. This reduces the number 
of large items to be bounded by 0(l/e 2 ). Then, with 1 — e loss, we apply 
arithmetic rounding on the reduced set of large items. Clearly the latter 
does not increase the number of items and each profit value is now equal to 
^-i for some i = A, A + l, 2 [X/e\ . By using this set of items with profits 
rounded by using geometric first and arithmetic rounding then, the size of 
set V has a bound of 0(l/e 2 ), and the overall time of the dynamic program- 
ming algorithm is 0(l/e 5 ). We call this combination a Serial Geometric & 
Arithmetic rounding technique. 

4.2 Adding small items 

In the following we show how to add the small items. First, with 1 — 2e loss, 
we reduce the number of small items to be 0{k/e) by using the Parallel 
Arithmetic & Geometric rounding (see Section l2.3j) . Then, for each pair 
(a, I) in the final list, fill in the remaining knapsack capacity c — g\L\(a,l) 

with at most k — / small items, by using algorithm Hz for kKP These 
small items yield total profit P H (c — g\n(a, I), k — I). By inequality © and 
by definition of small items, we have 

P H (c - g m (a, I), k - I) + eP H > OPT(c - g m (a, I), k - I), (16) 

where OPT(c — g\a(a,l),k — I) is the optimal solution value obtained by 
using at most k — I small items and knapsack capacity c — g\n(a,,l)- The 
approximate solution, a combination of large and small items, is chosen to 
yield profit P, where 

P = max {a + P H (c - g\ L \(a,l),k - I)} 

(a,l) 

By inequality ()16|) and since our algorithms considers all the "interesting" 
pairs (a, I) with 1 — 0(e) loss, it is easy to verify that P is 1 — 0(e) times 
the optimal solution. 

To summarize, the steps of the FPTAS are as follows. 

(S-l) Partition the set of items into "large" and "small". Apply the Serial 
Geometric & Arithmetic rounding technique to the set of large items. 
Apply the Parallel Arithmetic & Geometric rounding technique to the 
set of small items. 

(S-2) Solve for the "large" items using dynamic programming: generate a list 
of all "interesting" feasible combinations (a, I) of profit a and number 
I of selected large items. 

(S-3) For each pair (a, I) in the final list, fill in the knapsack by applying 
algorithm H 2 with the reduced set small items. 
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(S-4) Return the best found solution. 



Step (S-l) can be performed in 0(n) time. Step (S-2) takes 0(l/e 5 ) 
time. Algorithm Hz applied to the reduced set of small items runs in 
0(k/e) time [2|. In step (S-3) the algorithm considers 0(l/e 3 ) pairs, for 
each one performing operations that require 0(k/e) time. It follows that 
the overall running time of the algorithm is 0(n + k/e 4 + 1/e 5 ). The space 
complexity has a bound of 0(n + 1/e 4 ), since the space required by the 
dynamic programming is 0(X 2 (3) where A = 0(l/e) and (3 = 0(l/e 2 ). 

Theorem 9 There is a fully polynomial time approximation scheme for the 
k-item knapsack problem requiring O^ + k/e 4 + l/e 5 ) time and 0(n + l/e 4 ) 
space. 

Acknowledgments. Thanks are due to Klaus Jansen for introducing us 
to the k-item Knapsack Problem. We are grateful to the referees who pointed 
out some mistakes in the early version of this paper. 
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